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Information disclosed under the Access to Information Act 


1. Provide an overview of the A2SC-developed “Lighthouse” Risk 
Identification System (previously known as “Watchtower” ). 


2. Outline plans for an operational pilot to improve integrity and fraud 
detection in the global Student Permit (SP) caseload. 
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Built in-house and is readily adaptable to other IRCC business lines 


System can scan for almost all forms of data-detectable risk patterns, and be configured for specific risk types 
(e.g., misrep, organized crime, etc.) or countries 


Able to identify new risk patterns, thus enabling Risk Assessment Units (RAUs) to proactively select cases for 
verifications, based on previous indicators 


Enables RAUs and investigative teams to be more efficient and effective in identifying, validating and taking 
action on fraud and risk patterns 


Can contribute to improving processing speed of bona-fide applications 


Cost avoidance from fewer adverse events (e.g., each asylum claim costs the federal government roughly 
$16,000) 


Dovetails with existing intelligence-based risk detection approaches 
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* In July 2020, an early version of the system was briefly EN WU 
piloted for SP risk pattern/indicator detection, as a joint 
testing effort with teams from IN, DN, CN, IRM, and IPG. 


* Information on patterns and matching applications from 
the global SP inventory were shared with RAUs, who used 
it as one factor in their decision process for assessing Adverse rates rose to 1896 with Lighthouse, 
when verifications should be performed. compared to 8.596 without Lighthouse. 
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s.16(1)(b) 


Cohort level Applicant level 


Country level 


e RAO reporting * Emerging trends e Demonstrated fraud 
* Country conditions ° Program vulnerabilities ° Adverse outcomes DELHI investigation of SP applications 
e  Push/pull factors e Linkages between applicants e Mala fide intent 

| destined to Quebec DLIs (Feb 2021) 
e Global standard indicators 

St clas e Lighthouse was leveraged to support 

— a QA exercise relating decane: 

ten DLIs. 

e Stakeholders chose a single pattern 
to explore as a starting point to 
narrow their QA exercise looking into 
potential fraud. 

e Their exercise identified concerns in 
this caseload, | | 
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Risk 
matrices 


Lighthouse 


Case 
review 


Chinook 
RAO inputs above used to identify variables 


c .\ j and data fields for Lighthouse operation 
Training I — TTT e RAO feedback assists in refining business 
, rules for Lighthouse processes 
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The new pilot, to be launched in June 2021, will use Lighthouse to identify relevant risk patterns for RAUs on new 
SP applications to improve processing. 


Lighthouse will be one factor informing RAO decisions about when to conduct verifications of a client's documents. 
Two key objectives for the pilot: 


* Try a new business process: Before we can deploy the tool permanently, Ops needs to learn how to enable 
coordinated, Lighthouse-supported verification activities across the three Networks. 


* Better assess risks arising from the use of the tool: The July 2020 pilot was too short to allow Ops to identify 
unforeseen risks which may occur in the use of Lighthouse, and to implement mitigations for these risks 


TIMELINE AND TARGET VOLUMES 


June —Oct 18 weeks Lighthouse information will be provided on 50-100 applications/week (900- 
2021 (TBC) 1800 total) to IN, CN and/or DN RAUS, to support their decisions regarding 


whether to perform supporting document verification activities 
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System provides neutral, factual information to increase evidence available to officers 


* Lighthouse only provides information to support RAU decisions (not frontline officers) 
about when to collect additional evidence through a verification. 


e Pattern information is never based on sensitive/protected variables (gender, age, marital 
status, family name). 


* System identifies organized fraud, not broad risk profiles. 


e No decision automation. 


Lighthouse is never used directly in application decisions 


* Noapplication decision is ever based on information extrapolated from other applications 
or a past trends, only direct evidence collected from the specific applicant. 


The pilot is designed to avoid fettering 


* Frontline officers will not see Lighthouse pattern information and will not know which S 
applications are involved in the pilot. 
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"Do No Harm" design attempts to eliminate the potential negative impact of a 
"false positive" of Lighthouse. This is a key risk and bias mitigation. 


False positive = Lighthouse information leads to a verification being conducted that 
comes back as non-adverse (documents are authentic). 


We avoid harm in two ways: 


1) No processing delay. By moving verifications to earlier in the process, before 
frontline officers touch the application, we expect to avoid creating delays. 


2) Likelihood of approval is not negatively affected. The data shows that 
applications with a non-adverse verification generally have similar or higher 
approval rates than those that never had a verification performed. 
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s.21(1)(b) 
s.23 


TBS Directive on Automated Decision Making doesn’t apply 


But we still need to manage IRCC governance around algorithmic decision-making 


Need to continue to work with Legal 


Strong communication Plan required 
* The use of Lighthouse in this pilot may be covered in the media 


* Lighthouse-related information will be added to the information on advanced analytics in the upcoming 
Digital Transparency page of the IRCC website 


Business owner will need to be identified to strengthen accountability 
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Peer Review (Statistics Canada) Review Completed, 
Response Drafted 


Governance Working Group (A2SC, IRM, IPG, CN, DN, IN, Legal, Privacy, SPP, CMB, Conducted November 2020- 
Admissibility) January 2021 
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Finalize instructions to participating RAUs and frontline officers (A2SC, IN, CN, DN) To complete pre-launch 
= : D Eat —Á— — M oM—ÀÓoo 
t isk pattems to be used in pilot Netwone nM, Lea e i i 
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Includes monitoring of false positives for evidence of bias in specific patterns 
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Focuses on adverse outcomes (e.g. criminality), not approval/refusal of the application 


verall caseload adverse rate (about 096 to 396 for TR 


Profile-based patterns/trends (Macro-Patterns 
e Small number of high-volume patterns 
e Difficult to action in front-line risk assessment (i.e. too 
many clients match the criteria 
Adverse rates higher than baseline adverse rate, but 
often not high enough to guide action 
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s.16(1)(b) 


s.21(1)(a) 
s.21(1)(b) 


Lighthouse has been proposed for various other use-cases, including: 
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Engaged Stakeholders 


International Network (IN) 
Domestic Network (DN) 
Centralized Network (CN) 
Integrity Risk Management (IRM) 


Immigration Program Guidance (IPG) 


Legal Services (Department of Justice/LSU) 
Privacy (ATIP) 


Strategic Planning and Performance (SPP) 
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e A2SC goes to great lengths to ensure that Lighthouse is developed responsibly and does not 
introduce bias. 


e Steps include: 


I 


External review of Lighthouse by Statistics Canada 

2. Following best practices in data science to avoid bias and active monitoring of risk indicators 
3. Many layers of human review of the risk patterns to eliminate incremental bias 

4. Overall design centred on the interests of the client to avoid causing harm 


5. Active engagement with external stakeholders, including governance exercise in Winter 2020 


6. Follow the comprehensive ethics framework to govern AA work 
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Business Case for Investment in 
Advanced Analytics 
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Overview 


:ck provides a quick outline of the business case for an investment in Advanced Analyt 
from the Innovation Fund. 


It is based on prudent assumptions to avoid inflated expectations. 


In particular, the business case only estimates efficiencies to be made through automation of 
temporary resident visas (TRV), since the pilot provided a reasonable basis for estimating 
efficiencies. 


This business case does not include any efficiencies that would result from other lines of 
business included in the investment proposal, namely: 


e Student permits 
e Streamlining of low-risk citizenship applications 


e Automation of passport renewals 
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Assumptions for growth of TRV volumes 


nce, the deck assumes that the growth of TRV volumes will taper off to 
6 annually after 2020-21 and that volumes will plateau starting in 


* Faster growth would imply much greater potential efficiencies through advanced 
analytics. 


3000k 


2500x 


2000x 


1500k 


2014-15 2016-17 2018-19 2020-21 2022-23 2024-25 2026-27 2028-29 
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Deriving efficiency gains 


e Using their Time and Motion study, Finance estimated the average time required to proc - 
'RV application from start to finish. 


e From this, OPP subtracted the processing steps which would be automated by predictive 
analytics. 


e This represents the average time saved per application. 


e These time savings were used to extrapolate the efficiency gains in terms of FTEs based on 
automating roughly one-third of TRVs globally. 


e Out of prudence, this deck assumes that only half of those efficiencies would materialize. 


e This reflects, in part, the fact that a predictive model usually automates the simplest 
cases which are faster to process. 


e The FTE efficiencies were converted to salary dollars using the average salary of all staff 
involved in processing TRVs, including support staff 


* The average salary used was about S59K in 2016-17. 


* Since efficiency gains would mostly occur among higher-paid visa officers, this 
methodology includes some prudence. 
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Return on investment 


C 


tment proposal. 


e Cash flows would turn positive by 2020-21 and the initial investment could be recovered by 
2023-24, only four years after the investment is completed. 


Cumulative Discounted Cash Flows 
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s.21(1)(c) 
s.23 


Return on investment 


* Consi ering the uncertainty around long-term projections, we present a range of possi 


e Out of prudence, we have included more downside risk in the range. 


* The bottom of the range would see the initial investment recovered by 2027-28, eight years 
after the investment is completed. 


Cumulative Discounted Cash Flows (with Range) 
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Conclusions 


vhen focusing exclusively on TRVs, the project is expected to generate 
fficiencies to recoup the initial investment very quickly (four to eight 


e Efficiencies can be reinvested to reduce processing times in other business 
lines, strengthen program integrity or focus on more complex cases. 
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HIGHLIGHTS 


DESCRIPTION | ormative app 


i 
i 


DEPARTMENTAL | v Reduced processing times and Enhance client service 


PRIORITIES - 


OUTCOMES | increase (24/7 capac 


KEY | 1. 

DELIVERABLES | towards a global solution. 

| a. Predictive models will provide a daily analytics-based triage of all applications according to the 

level of risk. 

- b. Analytics-based triage will help identify cases for bulk approval without officer review. 

| 2. Deploy predictive models to support risk-triage for Citizenship and Passport applications, notably to 

| detect fraud; 

| 3. Develop forecasting models to forecast three TR volumes (series): overseas applications for Visas 
| (TRV), Student permits (SP), and Work permits (WP), as well as the volume of applications for 
| Citizenship and Permanent Resident Cards (PRCs). | 
| 4. Develop a Business Intelligence (BI) and Exploratory Environment (i.e. Advanced Analytics sandbox) to 
| support the project. | 
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Triage Only 


Decisions 


A4585065 5-000031 


Current State 
(All applications 
reviewed by 
officer, high 
approval rate) 


(Cases | 
for strea 
rocessing are 
not reviewe 
an officer) 


Existing steps 


New model steps 
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Current State 
(All applications 
go into the 
same queue) 


re 


Existing steps 


New model steps 
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» Rules used by the models will go through an extensive review process for potential 
discriminatory impacts. 


» A PIA will likely need to be conducted, however the level of impact (risk) is too early to 
determine until the assessment is underway and mitigation measures are identified. 
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September 2022 


A?SC will develop a 
detailed work plan and will 
report regularly (similar to 

TRV model) 


August 2022 
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Decisions 
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» 


» 


» 


September 2022 


A2SC will develop detailed 
workplans and will report 
regularly 
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Protected 


Business Requirements and 
Model Operations 


Name of the model: 
Spouse or Common-Law Partner in Canada (SCLPC) Advanced Analytics Pilot 


A2SC Business intake #: 
PR-FC-2021001 


Prepared by: 
Advanced Analytics Solutions Centre (A2SC) 
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1.3 Changes to Model Rules and implemented v1.2 

1.3 | A2SC | Jun 20, 2022 | SADMs (Operations and Policy) Approval for the model 
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1.3 A2SC April 25, 2023 Info about automatic GCMS Entry of triage results was 
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General Information 


Model Launch Approved by 
Branch/Division who is 
responsible for the Model 


DG of CN via Officer of Record 


- Solutions and Task Team 


: Centralized Network 


| Deputy Minister via memo 
.. IRM, IPG, and CN (in MPA) 


| This pilot seeks to streamline the eligibility assessment for both the Sponsor and Principal 
| Applicant of Spouse or Common-law Partner in Canada applications. 


| [ ITriage for workload sharing/distribution/classification 


 BXTriage for automated decision 


| Business rules suggested by users/SMEs 


: [ lRules generated by algorithm 


- &lCombination of business rules and algorithm 


| image and object recognition 


: Model Capabilities 
_ (Can select multiple items) 


: [ [Text and speech analysis 
_ ORisk assessment 


: [ IContent generation 


. &lProcess optimization and workflow automation 


| [ lOther (please specify) 


Summary of Initiative 


Background 


The ambitious 2021 Immigration Levels Plan, coupled with the operational challenges posed by 


COVID-19, has created an imperative for the Department to innovate rapidly. The Department has 
committed to a range of 300,000 to 410,000 PR admissions with an overall target of 401,000 for the 
current year. This necessitates a significant ramp up in staffing, training and final decisions (FDs) over 
the course of the year. The number of FDs per month is expected to ramp up significantly from 
26,000 in January to over 66,000 in September, according to the current operational plan. Without 
changes in how files are processed, significant challenges are anticipated in achieving these targets. 


In response to these challenges, and to complement Departmental efforts to continue processing all 


family class applications, Departmental officials were tasked with developing and piloting an AA 


model under a very short time horizon, in order to support 2021 Levels. This pilot aims to increase 


the output of cases and facilitate eligibility decisions in the SCPLC. SCLPC applications are generally 


3s evidenced by a historically | rate (i.e., 1 2019). 
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The proposed AA pilot seeks to streamline the eligibility assessment for both the Sponsor and 
Principal Applicant of SCLPC applications. The duration of the pilot will be for a period of 6-9 months, 
targeting an April 2021 launch. 


Objectives 


The objectives of this pilot are to support: 
i. reduce existing backlog of Spouse or Common-Law Partner in Canada applications 


ii. optimize the process and automate specific workflows for making positive eligibility 
determination and reduce processing times for all SCLPC files 


lii. increase operational efficiency while maintain program integrity 


IV. understand the potential impacts (legal, privacy, etc.)on the PR family class programs, 
and develop and implement mitigations for these risks 


Benefits 


If the Pilot achieves these objectives, it will facilitate expedited processing the FC inland applications 
to assist IRCC to meet processing target, and confirm SCPLC's potential to increase operational 
efficiency while maintain program integrity, as well as to improve client experience. The Department 
plans to assess and leverage the experience gained in the development of this model to extend the 
use of AA into other business lines. 


This is the first time that an AA model is deployed to expedite the processing of Permanent Resident 
(PR) applications, and as such, it represents a significant milestone. 


Key Decisions 


This section is to document the key decisions that are made for this initiative. 


Approved by | Key Decisions Notes or links 


Date | 
March 25, 2021 Data Executive | The committee agreed that the results 
Steering Committee | of the AIA of the FC model meet the 
ME (DESC) threshold of a level 2. NEM 
| | | EY Spousal Inland - April 2021- IMC and EXCOM ROD 
. April 1, 2021 Issues Management | Endorsement to move forward with |! (SECRET).docx 
Committee (IMC) . . pilot and proceed to ExCom | un 
- | bttos://ecdocs? ciec.ca/otcs/cs.exe/link/415903973 
. . 2 . . - AA Spousal Inland - April 2021- IMC and EXCOM ROD 
April 7, 2021 Executive . Members supportive of pilot | (SECRET).docx 


Committee (ExCom) : implementation. Brief up to MINO and 


wo, 


 https//ecdocs? ciac.ca/otcs/cs.exe/link/415903972 


: route outstanding AlAs that need to be 
_ published through governance. 
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Date 


April 13, 2021 


April 15, 2021 


. July 9, 2021 
August 23, 2021 


April 14, 2022 


May 19, 2022 


Jun 20, 2022 


Approved by 


DG of CN 


Key Decisions 


Approval of Officer of Record for SCLPC | 


AA Model v1.0 
. Approval the Launch of the Model: 


Deputy Minister 


| F-1158911 - Measures to Facilitate 


. SCLPC Processing 


ADM Ops and ADM 
Policy 


DG of CN 


- Approval of AIA and its release 


| Approval of Approval of Officer of 
. Record for SCLPC AA Model v1.1 


. "DGSC-ROD-2022-04-14.docx": 
| : /cs.exe/properties/457149365 


. Endorsement to move forward from 


: pilot to permanent. 


DG of CN 


| Approval of Approval of Officer of 


_ Record for SCLPC AA Model v1.2; 
| implemented on May 30, 2022 


Senior ADM Ops 
and Policy 


Model Changes 


| Approval the Transition of the Model: 
. F-1224710 - from Pilot to Permanent 
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Notes or links 
` 1. Final Approval Memo to CN - Officer of Record 


| https //gcdocs2.cige.ca/otcs/cs.exe/link/396006493 


. "e-approvals": 


. "e-approvals": 


: Approvals 


^ 


oroanerties/ás0 iod» 


| "SADM Approval Package F-1224710": 
g fes eve foroperties/4547 79490 


The model was approved by the Deputy Minister for launch. Any changes (i.e. adding, removing and 
modifying rules) to the model rules will be approved by the Director General of CN through the 
Officer of Record. If any risks that are identified to be addressed immediately, will follow the new 
AA Governance Process developed by A2SC and approved by DGs. 


| SCLPC AA Model v1.0 DG of CN 


SCLPC AA Model v1.1 DG of CN 


| SCLPC AA Model v1.2 DG of CN 


April 13, 2021 


Aug 23, 2021 


May 19, 2022 


R1-R7 
Implemented on April 21, 2021 


Added R8 and R9; 
implemented on Jun 4, 2021 


Added R10, R11, R12; modified R2 
implemented on May 30, 2022 


Model Details 


Business Workflow 
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The processing office will continue their regular intake functions, including carrying out a document 
completeness review, for both the principal applicant and sponsorship applications. The AA model 
will be run at a regular interval following the completeness check, creation and promotion of the 


application. 


How the Model Works 
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Intake 


The model identifies low-complexity SCLPC applications, for which the eligibility of both the Sponsor 
and Principal Applicant (PA) would have a high likelihood of being approved in normal course of 
processing, and can therefore be triaged by the model and administratively passed. The model uses 
rule to place SCLPC files into either a "Green Bin" (GB) or a "Standard Bin" (SB) which results in the 
following actions: 


Applications placed into the "Green Bin" will receive a positive eligibility determination of Sponsors 
and Principal Applicants based solely on the model's assessment. The model will not assess eligibility 
for dependants in the GB, therefore, the eligibility of dependants to be processed by officers as they 
are currently processed. Admissibility assessments for applications placed into the "Green Bin" will 
be completed by officers, as per standard procedures. 


Applications placed into the "Standard Bin" will have both the eligibility and admissibility of Sponsor 
and Principal Applicants assessed by officers, as per standard procedures. While these applications 
are considered to be more complex, given the overall high approval rates for this line of business, 
applications in the "Standard Bin" are still highly likely to be approved. 


The model does not make any form of negative recommendation on an application, and that no 
applications will be refused by the model. 


Use of the Model 


Who will use the model? 
The staff at the centralized network (CPC-Mississauga) will use the model output for processing. 


For files that are in GB, a positive eligibility decision for the sponsor and the principal applicant is 
made by the model. Even if in GCMS it appears that the employee made the decision, the 
accountability and responsibility rest with the Director General, who signed the Officer of Record 
memo. Following those GCMS steps, the file will move forward for standard processing. 


For files that are in SB will be processed by officers as usual. 


All applications will still require assessment, as per standard procedures for eligibility of dependents, 
and all elements of admissibility, including criminality, security, and medical, for the Principal 
Applicant and dependents. 


A4585070. 8-000061 


eq 


Immigration, Refugees Immigration, Réfugiés 
fi + i and Citizenship Canada et Citoyenneté Canada 
Information disclosed under the Access to Information Act 
s.1 6(1 )(b) L'information divulguée en vertu de la loi sur l'accès à l'information 


Protected B 


instructions {PDis) 
instructions on how to use the model as well as to mitigate risks around fettering officer decisions. 


Clear program delivery for officers have been published on connexion to provide 


Officers are instructed not to review “Green Bin” applications for eligibility, since this determination 
is made using the AA model, however they retain the discretion to re-evaluate the eligibility 
determination of a “Green Bin” application, as necessary. 


GCMS entry 


Started on April 28t, 2023, GCMS automatically (according to CR# 838409?) assigns GCMS Org ID 
based on the triage results provided by the Advanced Analytics Solutions Centre (A2SC). 


Bin Org Name Org ID Notes 
Green 
Green + Dep 
Standard 
Green QA The QA ID was used from the launch of the pilot on April 


21, 2021 and stopped on April 11, 2022, requested by IRM 
and CN to implement a blind QA system. 


Business rules for GCMS Automatic data entry: 
System must automatically assign the ORG ID based on its corresponding number only when the 
following conditions are met: 

e Categories must be FC1 or FCC 

e Sponsorship Eligibility Assessment field is equal to ‘Not Started’ 


b Category Eligibility Activity Type 


[ri —  [iigbiiity-FCSponsorship 
[FC — à Eligibility - FC Sponsorship 


© 
gibility Activity Type 
ram RE 
FCC Eligibility 
e Application Status field is ‘Open’ 
e Final Decision is blank 


If any applications cannot be automatically entered due any unexpected reasons, A2SC will provide a 
list of these applications with CN for manual data entry. Regular weekly reports will still be provided 
with CN for a required timeframe for QA purpose. 


CN will receive a notification from A2SC after the automation is completed every week. 


! Prior to April 28, 2023, CN (CPC-M) associate different organization IDs with applications that have been triaged by the 
system for tracking based on the triage results. 


? "Detailed Requirements for CR #838409 — Automatically assign bin numbers and User ID based on advanced analytics 


triage.docx": https: 
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s.16(1)(c) 


How the model is developed 
How the rules were developed 


The model was initially developed based on a set of evidence-based, data-driven rules. These rules 
reflect factors that correlate with approvals. The rules were then further interpreted and validated 
by subject matter experts, in order to ensure that the appropriate justifications are documented and 
to address possible concerns about bias, including removing rules that were controversial. 


The proposed model was tested on a data set of 40,343 inland spousal applications that were 
processed in 2018 and 2019. When tested, the model predicted that roughly 59.396 of the caseloads 
would be classified as "Green Bin" and the remainder were classified as "Standard Bin". The 
predictive accuracy of this test was 99.396, meaning that only 0.796 of applications that were 
predicted to be positively triaged by the model, did not result in a positive eligibility determination. 


See details of the approved model rules in annex A. 


Constraints 


. This has been communicated with senior management and 
stakeholders and documented in various documents such as memo, risk register and risk mitigation, 
etc. 


Internal Peer Review 
During the development of the model, multiple data scientists reviewed the modelling to ensure the 
methodology is sound and logical and it is designed to achieve the expected outcome. 


Data quality and Bias 

When designing the model, data scientists reviewed the key data elements that are relevant to the 
program, such as marital status, age and client history, etc. and made efforts to identify any data 
discrepancies and quality issues and how it affected modeling. There were no major concerns, some 
minor problems were discovered but fixed (some of them are not used for the model). This was 
documented in A2SC's issue log, which is part of A2SC's normal process. 


An analysis of the characteristics of the overall population of the training data was also performed 
including gender, age, sexual orientation. 


Privacy and security 
The team consulted the privacy and IT security experts at IRCC at the design stage to ensure any of 


the privacy and security principles were considered in the design stage and protocols were followed, 
no concerns were identified. 
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Operations of the Model 


Model Input 


Updated on Nov 10, 2022 requested by CN-CPCM 


Application Primary Office is equal to / is in CPC Mississauga 


The model runs after completeness check, creation and promotion of an 


and 
.. Application Category is equal to / is in FC1; FCC application. *CN only transfers digitized GB files IN 
and 0149069334341 — Incomplete apps 
-= Application Subcategory is equal to / is in CDA; PP O261799254484 — Incomplete apps 
and O140635752433 - Incomplete apps 
Sponsorship Eligibility Assessment is equal to /is in Not Started — 0261799254569 — Incomplete apps 
and 
(PA) Eligibility Assessment is null means: file is rejected due to incomplete application is submitted, which is 
and identified at the file creation stage prior model triage. 
Application Status is Open 
and 


T n m Add the following two Org ID only to the output report. 
Application Assigned To is equal to/ is in Unspecified Org ID O268073013482 - Returned from digitization 


and l l l means: files are returned from 3rd party and completed digitization, docs are 
Org ID is not To is NOT in 0149069334341, or 0261799254484 _ uploaded (CN transfers these cases to IN) 


or 0140635752433, or 0261799254569 
and — _ Org ID 0269754626955 -Not digitized 
Org Disassociation is blank means: files are neither digitized nor uploaded, these files are usually paper 
applications and will remain with CN for processing) 


. The model runs after completeness check, creation and 


promotion of an application. 


*CN only transfers digitized GB files IN 


aes: 


Add the following two Org ID only to the output report. 


and 
“ (PA) Eligibility Assessment is null 
aad Org ID 0268073013482 - Returned from digitization 
E: Application Status is Open means: files are returned from 3rd party and completed 
and digitization, docs are uploaded (CN transfers these cases to IN) 


f Application Assigned To is equal to / is in Unspecified 


Org ID O269754626955 -Not digitized 
means: files are neither digitized nor uploaded, these files are 
usually paper applications and will remain with CN for processing) 


The model runs after completeness 
check, creation and promotion of an 
| and application. 
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Sse 


sons *CN only transfers digitized GB files IN 


MESS 


pass 


: and 
. * Application Assigned To is equal to / is in Unspecified 
| and 


: T Paper File Location is equal to / isin Registry; Inland - ready for decision maker; Inland 
- Review- Pre-Screened Pending; Inland Pending Review; Inland Inventory; SCLPC - Digitized; 
. Digitized Inland Review - Pre-Screened Pending; Digitized - Inland - ready for decision maker 


Model Output (in Excel) 


Updated on April 11, 2022 requested by IRM and CN for implementing blind QA 


App PA . PA SPR SPR App Lock-In PA Corr 2 SPR Corr  Dest _ HasorgiD  HasorgID Paper file. Bin CN : Scored On Bin (hidden in 
# : UCI  HH UCI HH Date , - : | 02680700 02697467 Primary : . Date output report for 
l uc UCI | Language Language Prvnc 3482 -6955 - Location : | IRM QA) 
. English 2 Alberta Y ON | SCLPC - GB | yyyy-mm-dd GB 
| | | | - Digitized : 
French Ontario N Y Registry SB yyyy-mm-dd SB 
SB GB-QA 
GB-Dep GB-Dep 


| App 8 PAUCI PA SPR SPR  Applock- PA Corr | SPR Corr Dest  HasorgID  HasorglD Paper file . Bin . Scored On 

|  househol UCI  househo  InDate 0268073013 0269754626 Primary Date 
ss. due dog PERS i UE Re SE on e 
© FOOXXXX  XXXXXX XXX (XXX XX | yyyy-mm- — English XXXX Alberta — Y N | SCLPC-Digitized Green yyyy-mm-dd 

. FOOXXXX — XXXXXX XXX (XX XXX | yyyy-mm- French XXXX Ontario N Y | Registry Standard  yyyy-mm-dd 

: : : dd : : 


Frequency (and Volumes} 


The 1* report was produced on April 21 to launch the model, with approx. 5200 triaged applications. 


Weekly reports started with the 2 run on May 10, 2021 with 347 triaged applications. It was 
estimated that the model will triage approx. 500-600 applications each week. 


Distribution 
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The weekly reports (GCDOCS links) are distributed from A2SC's general mailbox 
IRCC.OPPAZSC. REPORTS-RAPPORTS. CSA2PRO.IRCC@cic.ac.ca to the distribution list 


Recipients of the reports (requested by business): individuals at CN-NHQ, CPC-M, IRM and IPG 


A reminder of proper use of the model (PDI link) and privacy and safeguard is also included in the 
email. 


Retention and Disposal 


ee ea Retention Period — — Notes LL 

Model Code < Source control indefinitely The current model in production is retained 

- (managed by IRCC) - indefinitely. When the model is updated 

and Data Scientist's . (retrained, new rules, any other modification that 

. Work Station . would affect it in production), then the previous 

: : . version is retained for 10 years. 
Model Output — Officer — Cloud — — .— l0 years ——— _ Reports for Officers: Saved in the Protected B 
report - GCDOCS 2 years | cloud and automatically deleted after 10 years 


. (moved to cloud due to space issues in GCDOCS). 

| The copy of the report in GCDOCS will be deleted 

_ after 6 months. The retention needs will be 
EE : reassessed during the pilot 


Model Output — Legal : Cloud 10 years 
report : GCDOCS 2 years 


Quality Assurance 


1) QA by IRM 


In order to reduce program integrity risks, the Department (IRM branch) has enacted a continuous, 
targeted QA program based on a random sampling of “Green Bin” applications. Officers at CN (CPC- 
M) will manually assess the eligibility of the sponsor and principal applicant in the QA sample. The 
results of the QA exercise are captured in the PI tool and QA reports (by IRM) will be used to monitor 
and improve model performance. 


e QASize 


As agreed with the stakeholders, A2SC will confirm the QA number requested by IRM each week for 
the first few runs until a fixed QA number can be confirmed for each run. 


In November 2021, IRM requested A2SC to update the QA sample size from a numeric 10 to a sliding 
scale of 2.43% of the GB by using conventional rounding rules (i.e. .1 to .4 = round down, .5 to .9 = 
round up) for each run. This will continue going forward until requested otherwise. The requested 
QA change were implemented on December 6, 2021. 


e Blind QA 
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In April 2022, IRM and CN updated the QA plan, as a result, a blind OA system was implemented on 
April 11, 2022. 
Requirements details from IRM and CN 
e “Green — QA" files are to be re-triaged to the Standard Bin. As such, they will not appear as QA files, 
nor will they be distinguishable from Standard Bin files, in CN's records or in GCMS. CPC-M will no 
longer be aware of which files are QA. 


Processing officers will no longer be able to determine whether a given file is QA. 
e Though re-triaged to the Standard Bin, QA files will continue to be coded by OPP in a way that they 
will still be identifiable to IRM on the weekly output report. IRM's analysis of QA files will continue as- 
is. 


Impact of the Blind QA 

1) there won't be any new entries of QA cases displaying in GCMS after the implementation. 
Therefore, reporting of QA cases in GCMS Answers won't be available. 

2) A2SC can provide App numbers of QA cases; IRM also tracks QA cases from A2SC's weekly report. 

3) If any parties want to track QA cases in GCMS Answers, the old QA org ID can be added in GCMS 

anytime (i.e. after the eligibility decision or final decision is made). However, it is not A2SC's decision 

and A2SC is not able perform this task. 


2) Frontline feedback and monitoring on GB 


In order to capture feedback from the frontline officers and ensure any issues are timely flagged for 
monitoring and model performance improvement, CPC-M officers are instructed to tag an 
organization ID . to the GB cases that may have eligibility concerns at the 
admissibility assessment stage in late August, 2021. These applications were flagged does not mean 
the model triaged them unsuccessfully because an applicant could be eligible at the time of triage 
but new info is added after triage and model is not able to detect, etc. The eligibility decision on 
these applications could still result in a pass. 


Heporting 
Report Details By Notes 
Weekly model output table GB size and SB size and A2SC Weekly progress reports were shared with networks. 
(for operational volume Volume, completion of 
m ALES SU LS Se ee ee ee eee eee eee 
GBA + report GB, SB, client's gender, age, A2SC An initial report was produced by ST and A2SC, and reviewed by 
(for unintended bias) etc, potential impact of the GBA+ team, SPP, IB. A follow up report was conducted in Mar 2022. 
— tii MAE rules, q———— 75 
QA report IRM An initial report was produced in June 2021. Weekly report started 
(for program integrity) in Feb, 2022 


Stakeholders and Key Activities 
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Branch Key Activities/Products Contact 

E OPP (A2SC) Model development and delivery, technical documentation, = Jae-Jin Ryu 
and weekly model triage. Taylor Williams 
Work with key stakeholders to develop business Na Guan 
requirements, and monitor model performance. David Son 
Coordinated completion of AIA Joseph Bogaard 

eat Sg TR EEUU YN TS eee ae 
provide design input and develop model rules with A2SC; Candace Mantha 
initiative coordination and develop and maintain mitigations. 

"ec B US E =. PE S EAE EN = s a = 
indicators, risk registry. Monitor QA results and provide Ryan Kaiser 
feedback with A2SC for model improvement. Parveen Sandhu 

" : NCC oo IE EM——— EE 
weekly model triage output, provide feedback with A2SC on Elise Pineau 
the use of the model. Perform QA exercise, provide feedback Chrissy Haley 
with A2SC for model improvement. Jessica Thomas 

"— MN EINER Nicole Mason. 

IPG Develop PDI and instructions for operations. Update Julie Antiporda 
application forms. Conduct Privacy Impact assessment with Alexandra NgYong 
ATIP Isabelle Corbin 

"cou T " SGH ee A 
and provide input/comments on GBA+ analysis, AIA, etc. Benoit Richer 

— ee : ae c " mes = 
mitigations. Provide legal Opinions. Marta Lamosova 

" e ce Iu MEE de 
mitigations. Conduct PNA and MPA. 

… ni. {oo ne 
when applicable. Transparency page, notice for clients, Natalie Pierce 
comms implications in briefing materials. 

a Re o a e ee m " 
Management — G&Á  —— —— s 
IB Patriciate GBA+ workgroup meetings, provide input in Sonja McKay (IB) 
GBA+ conducting GBA analysis and monitoring plan. Yvonne McKinnon (GBA+) 


Governance Check List 
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The following items have been identified by stakeholders and completed for governance purpose. 


- Documents Notes Link 


Accountability Approval - XMEMO for Approval of the model DM Memo and Appendix DM memo package 


- Appendix A Rule justification 
Dl Officer of Record Aa ae 
- Appendix B TBS requirement 


- Appendix C Risk and Mitigation Registry - D CENSURE 


XI Risk Register  Bisk registry 


n o aoa ee "MR 


: ODAH — XGBA+ Initial and Follow up | GBA+ analysis 

 Explainability Training Program Delivery Instructions —————0- EO 
| . Rule Justification | 
Privacy | Assessment | X Privacy Needs Assessment PNA 


[ |Privacy Impact Assessment 


VER A 
. eG Model Privacy Assessment 
- Notice A web or other notice for historical . -Historical Applicants: Whose data was usedto : Notice for historical 
| .. develop the model | Clients 


-clien | 
Res . -Inventory/Current Applicants: Who have 


A web notice "After You Apply" for : applied and will be triaged by the model | A web notice " for 


: inventory clients | - Future Applicants: be notified at the time of | inventory chents 


collection according to s 5.2 of the Privacy 

Act . Notice on Application 
. Forms was added in 

| | | . the summer 2022 

| | FORM IMM1344 


. DÜPrivacy notice on Application Forms for 
. future clients 


- Security | Security | [ ]Security authorization | The model runs in an environment which is 
pre-authorized by IT security. 


Transparency . . DX Transparency Page 
| | . [XI Algorithm Impact Assessment 
Peer Review 


Other - Monitoring Plan for the use of the model . Monitoring the use 
| - Initiative tracker - initiative tracker 
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IRCC has enormous data holdings but struggles to turn this into real-time risk intelligence for 
decision-makers. To address this challenge, IRCC developed a new data mining tool, Lighthouse 
that leverages advanced analytics te s to identify applications that may be high-risk. 


Lighthouse dovetails with existing intelligence-based risk detection approaches and can 
comprehensively analyze large volumes of data and identify patterns that may point to high-risk 
clients with adverse characteristics, such as criminal activity and misrepresentation. 
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decision-makers. 
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Within global caseloads, Lighthouse automatically and systematically identifies pockets of connected 
applications with a high incidence of adverse events 
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iority to use a: IRCC — has the required auto" ties in accordance with the 
M igre ra ati on Co on it tro! and Security Management personal information bank, PPU068, as described 
under "consistent uses” that include the detection, Suppression, and prevention of | igs tebe 
offences, for quality assurance purposes, and to determine an indiv idual's admissibili ity. IRCC may 
also use this personal information with computer analytics to support research, statistics, prog ram 


and policy evaluation, internal audit, compliance, risk n 1anagement, strategy development, and 
reporting 


Not applicable. 
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ment with key partners, such as Statistics Canada. 
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Privacy / 
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Assessment July 2022 (TBC) 


» Presentations to Corporate Governance August 2022 (TBC) 


DM Memo August 2022 (TBC) 
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Within global caseloads, Lighthouse automatically and systematically identifies pockets of connected 
applications with a high incidence of adverse events 


MINE, DETECT & Report ASSESS DISTRIBUTE 
Uses data mining Are the risk patterns free of RAUs, CMB and other risk VIEN 
techniques to find aii plas as intended? _ practitioners review the framework 
possible risk patterns and Does the summary report : risk patterns identified in ensures feedback loop anc 
identify those that are provide officers with the the report, and conduct regular evaluations are 
strongest. Automatically core information needed to further assessment and conducted to strengthen 
generate a summary report. quickly assess a trend? validation of related the tool's performance. 
applications. 
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Lighthouse dovetails with existing intelligence-based risk detection approaches. 
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Once patterns have been identified, a variety of supplementary information is supplied to users to help 
contextualize a pattern: 


y: How many applications meeting the pattern have adverse info? What is the breakdown 


> Text: 
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» The AIA for Lighthouse was developed by 
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“The impacts of automating an administrative decision are classified into 4 levels, ranging from Level I (little impact) to Level IV (very high impact). 
impact levels are distinguished based on criteria of reversibility and expected duration: automated decisions with little to no impact are 
reversible and brief, while those with a very high impact are irreversible anc perpetual. 


. Decision-making continues to rest with officers. 


» IRCC's primary data partners, Global Case Management System (GCMS) and Enterprise Data Warehouse 
(EDW), follow processes to ensure that only relevant and reliable data is used. GCMS has rigorous 
processes for ensuring the reliability of data used in application processing. 


» A governance framework for Lighthouse exists that outlines how the system will be managed and 
maintained responsibly. 
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Mitigations 


applications are assessed against historical patterns of 
concern at the very front of the process, when an 
application is first received by IRCC. 


Refusal Likelihood — to avoid fettering of officer 
decisions and any negative impact on the client via a 
reduced likelihood of being approved, the process is 
designed to ensure that decision-making officers will 
not have access to Lighthouse-derived risk information. 


Inequality (e.g., racism, sexism, etc.) - Lighthouse 
treats applications from different countries, gender, and 
age groups in a manner generally consistent with the 
historical risk of adverse information being found in 
applications from those groups. 


Lighthouse was "trained" to identify patterns of concern 
on 799,000 student permit applications received by IRCC 
between January 1, 2019, to December 31, 2020 
(“training set") to evaluate the historical bias assessment. 


Examined the following indicators: 
Gender: Rate of Adverse Applications vs. Rate of Lighthouse Pattern Matches 


Country of Residence: Rate of Adverse Applications vs. Rate of Lighthouse 
Pattern Matches 


Age: Rate of Adverse Applications vs. Rate of Lighthouse Pattern Matches 


Based on the test set analysis, applications matched by 
Lighthouse aligned relatively well with the underlying risk 
distribution in the general caseload. 


IRM will continue to monitor and review the tool's output for 
any real or perceived bias. 
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IRM, 


in collaboration with A2SC and the integrated Network, will: 


'eekly reports provi ided by ASC. The review will 
act as an ind pain ion to screen His any v uni nine bias in the tool's output. 


» Monitor and report to A^SC on the suitability of the risk indicators based on various factors, such 
as context, recency, frequencies, and potential biases, when required. 
o Afeedback mechanism on risk information will be provided to the Networks and other 
relevant stakeholders so that continuous engagement and expertise are shared to ensure 
configurations and tool are performing adequately and accordingly. 
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GBA+ and MPA August 31, 2022 
AIA September 2, 2022 
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Officers i in rid field use risk indicators to inform decision-making based on known eligibility and 
admissibility fraud trends 
* Risk indicators are developed via local knowledge, targeted integrity exercises, business 
intelligence from partners, large-scale trend analysis 
e Risk assessment units (RAUs) across the integrated network undertake anti-fraud verifications on the 
basis of officer referrals. Officer referrals are based on experience, judgement, knowledge of local 
conditions 
e Verifications involve open source intelligence, phone verifications, verifications with third parties, 
site visits/interviews 


* Nomeansto osos vast E duis holdings in GCMS to responsively develop risk-based processing 
guidance 

* Nosystematic way to share case-specific risk information to individual officers 

* Models and assumptions which govern automation, bulk processing, and other facilitative processing 
approaches cannot respond to evolving risks in real time 
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‘intégrité d 
Utiliser les tendances dans nos données pour détecter la fraude jusqu'alors inconnue en permettant des 
connections que les officiers individuels ne peuvent observer 

L'identification des tendances de risque pour les dossiers ouverts en temps réel et basé sur des données 
Une meilleure détection du risque permet au Ministere d'experimenter en confiance avec les approaches de 
traitement facilitatrices pour les clients à faible risque, en sachant que les risques sont controlés. 


mment on y arrive? 


Outil d'analyse d dances en matiere d'intégrité (OATI) est un outil d'exploration de données développé 
par l'IRCC qui utilise des analyses avancées (AA) afin d'extraire des tendances de risque ou de dans une ligne 
d'affaire donnée, selon les données du SMGC. 

Mesurer ces tendances permet à l'IRCC de définir précisément la tolérance de risque, et d'adjuster cette 
tolérance selon l'évolution des tendances. Alors que nous devenons plus sophistiqués dans notre approche au 
risque, nous pouvons mieux calibrer le niveau d'effort dans le traitement des cas, selon les risques acceptables 
vs. inacceptables. 

En avance du MPN3, la capacité de l'OATI à identifier activement les demandes requiérant une revue en 
profondeur, versus celles qui représentent un plus faible risque, est fondamental pour soutenir la vision de 
l'IRCC du traitement individualizé des voyageurs. 


A4594152 3-000103 


Immigration, Refugees Immigration, Réfugiés 
it de i and Citizenship Canada et Citoyenneté Canada 
Information disclosed under the Access to Information Act 
L'information divulguée en vertu de la loi sur l'accès à l'information 


s.16(1)(b) 


Exemples: 


esure de renvoi 
\ Fraude de documents vérifiée 


Interdiction de territoire 
Fausse déclaration 
Alertes d'information 


infraction d'immigration au Canada 


Exploration d 


données 


e 
5 
p 


Dn AU AAA A AA VAE A AP MA A A III NRI HD p 
I 
PE no AAI SAA AR SAA AN RAR AR RAA SAA, BAP SAR KR) SAR RA) NAR VAR, RAE XAR, A RR) SAR 200 O09 


oe 
ve 


A4594152, 4-000104 


Immigration, Refugees Immigration, Réfugiés 
it de i and Citizenship Canada et Citoyenneté Canada 
Information disclosed under the Access to Information Act 
L'information divulguée en vertu de la loi sur l'accés à l'information 


5; verificat 
wait initi 


a 


RAU, accruing additional 
ductive time 


WA RAMUS DONS DS GAME SE RE Zee Chee RS eR Iul CS SR SR iecur CS DNS SRE SNS Mere CS Cee SSI i 


" uns Se he eee oes uus eee NM E Rese e v Cd 
| um MASSE MERE EMIL GEE CANNE HE ES Se dd um zoe 


processing times, as 


Existing steps 


New model steps 


Changed step 


A4594152, 5-000105 


Immigration, Réfugiés 
et Citoyenneté Canada 


Canada 


Ip 


, Refugees 
h 


ion 


t 


Immigra 
and Citizens 


Information disclosed under the Access to Information Act 


à l'information 


i sur l'accès 


= rtu de la lo 


guee en ve 


L'information divul 


t 


ion Wi 


Icat 


if 
icer experi 


r Verl 


ion rely on off 


U 
xd 


ity Exerc 


d Integr 


S 


Ime 


t 


o0 
= 
un 
Y) 
U 
c2 
O 


indi 


rse f 


A4594152_6-000106 


Ce qu'il fait: 


Y Personnalisable, flexible, et 
adaptable à n'importe quelle 
ligne d'affaire 


Y |dentifie les liens forts aux 
dénouements défavorables pour 
enquéte plus approfondie 


Y Analyse presque toutes formes 
de tendances de risque 
identifiable par les données 


v Configuration pour certains types 
de risque: fausse déclaration, 
verifications qui ont décelées la 
fraude, renvois, exécution de la 
loi, criminalité 

v Soutien l'automatisation, le 
traitement en groupe, et autres 
Innovations 
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Ce qu'il ne fait pas: 


x 


Identification des demandes à faible 
risque pour traitement 
Automatisation des prises de décision 
Partage de l'information directement 
avec les agents décideurs 
Identification des demandes pour 
refus a base de ‘bonne for (e.g. 
intention de quitter le Canada, fonds 
disponibles) 

Baser les tendances de risque sur 
des demandes aux taux de refus plus 
élevés (sans infos défavorables 
documentées) 

Détection de toutes les tendances de 
risque 
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» IRM completed a GBA Plus analysis in May 2022. The analysis assessed the potential for bias and differential 


impacts when using the Integrity Trend Analysis Tool. 
» If approved, the methodology for identifying institutional bias (Differential Outcomes Framework) will 


provide guidance to help ensure any bias is not perpetuated. 
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Mitigations Data Potential differential outcomes 
di icati ha are assessed aga inst hi stor ical patterns of 
concern at the very beginning of the process, when an 
ite. ° — === 
— M ENT 
Refusal Likelihood — to avoid fettering of officer : PP Ne es 
instances of fraud/criminality/malfeasance, not suspicion. 


decisions and any negative impact on the client via a Often, once significant fraud trends are interrupted, refusal 


reduced likelihood of being approved, the process is 
designed to ensure that officers continue to make 
informed decisions based on the information provided 
in the application and the verification results. 


inequality (e.g., racism, sexism, etc.) — applications 
from different countries, gender, and age groups are 


rates subsequently decrease, as knowledge amongst bad 
actors spreads that fraudulent applications will be 
intercepted. 


Characteristics of applications identified through ITAT aligned 
with the underlying risk distribution in the general caseload. 


treated in a manner generally consistent with the 
historical risk of adverse information being found in 
applications from those groups. 


IRM will continue to monitor and review the tool’s output for 
any real or perceived bias, and adjust the tool's parameters if 
unjust differential outcomes are observed. 
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Evaluation de - L'évaluation de incio 5 m A) Plan de l 'AQ En collaboration a avec cle DGPRO « et tler réseau u intégi ré, la 
l'incidence Une ere del nadane algorithmique (E A) GRI devra : 
algorithmique pour l'outil a été élaborée en consultation avec des Mener des examens Varus des un 
intervenants partout au Ministere. L'outil a recu un hebdomad 
niveau d'incidence de 2 sur 4 (modéré) servira de fonction de Ae pour PR tout 
O———————— biais non intentionnel dans les résultats de l'outil. 
L'outil n'est pas un outil décisionnel. La prise de Surveiller la pertinence des indicateurs de risque et en 
décisions continue d'étre la responsabilité des faire rapport au DGPRO en fonction de divers facteurs, 
agents. Les données utilisées sont en provenance de comme le contexte, la mise à jour, la fréquence et les 
l'SMGC, qui comporte des processus rigoureux pour biais potentiels, au besoin 
EN M "MM 


Consultation externe: Le SCT exige que les outils 
d'analyse avancés de niveau 2 soient soumis à un 
examen des pairs. À ce but, Statistique Canada a déjà 
complété un examen de l'outil. 


ll existe un cadre de gouvernance pour l'outil 
qui décrit comment le système sera géré et 
maintenu de façon responsable. Le cadre d 
gouvernance portant sur les outils 

d 'analyses avancés a été suivi pendant le 
developpement et pilotage de l'outil. 
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other AA tools 


A4594152 11-000111 


Immigration, Refugees Immigration, Réfugiés 
it de i and Citizenship Canada et Citoyenneté Canada 
Information disclosed under the Access to Information Act 
L'information divulguée en vertu de la loi sur l'accès à l'information 


Approval to launch 


Concurrence with phased roll-out timeline 
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L'évaluation de l'incidence algorithmique (EIA) pour l'outil d'analyse des tendances en matiere d'int 
a ete élaborée par la Gestion des risques pour l'intégrité (GRI) et irection generale de | C 
et du rendement des opérations (DGPRO); les intervenants d'IRCC ont été consultés. 


*Les repercussions de l'automatisation d'une décision administrative sont classées en quatre niveaux, allant du niveau I (incidence minime) au niveau | 
(incidence tres élevée). On distingue les niveaux d'incidence en fonction de critères de réversibilité et de durée attendue ; les décisions automatisées 
ayant peu ou pas d'incidence sont réversibles et brèves, tandis que celles ayant une incidence très élevée sont irréversibles et perpetuelles. 
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° iban: use (is indicators to inform decision-making based on known eligibility and 
admissibility fraud trends 
* Riskassessment units undertake anti-fraud verifications on the basis of officer 
referrals. Referrals are based on experience, judgement, knowledge of local conditions 
* Verifications involve open source intelligence, phone verifications, verifications 
with third parties, site visits/interviews 


° No: means to es à vast ids holdings in GCMS to responsively develop risk-based 
processing guidance 

* Nosystematic way to share risk information to individual officers 

* Risktrend information not systematically available to officers 


° Réal -time, en bind identification of admissibility risk trends in live case loads 
* Maximize value of risk assessment units efforts to uphold program integrity 
* Aresponsible risk backstop that facilitates innovative processing approaches 
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s.16(1)(b) 
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ithouse détecte les profils de risque au niveau micro et macro 


Cette image montre comment Lighthouse détecte les profils de risque au niveau micro et macro. Les microprofils sont rattachés à des clients à 
risque élevé, et ils sont déterminés par des liens solides entre d'autres applications, E | | m m 
La plupart des applications sont au niveau de la base, puisqu'elles sont à faible risque. 
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s.16(1)(b) 


Within global caseloads, Lighthouse automatically and systematically identifies pockets of connected 
applications with a high incidence of adverse events 


MINE, DETECT & Report ASSESS DISTRIBUTE 

Uses data mining Are the risk patterns free of RAUs, CMB and other risk VIEN 
techniques to find aii plas as intended? 7” practitioners review the framework 
possible risk patterns and Does the summary report "risk patterns identified in ensures feedback loop anc 
identify those that are provide officers with the the report, and conduct regular evaluations are 
strongest. Automatically core information needed to further assessment anc conducted to strengthen 
generate a summary report. quickiy assess a trend? validation of related the tool’s performance. 


applications. 
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What it does do: 


Y Customizable, flexible and 
adaptable to any IRCC business 
line 

Y Identifies strong links to adverse 
outcomes for further investigation 


Y Scans for almost all forms of 
data-detectable risk patterns 


Y Configured to specific risk types: 
misrepresentation, adverse 
verifications, removals, in- 
Canada enforcement, criminality 


Y Supports effective processing 
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What it does not do: 


Identifles low-risk applications for 
processing 

Automates decision making 

ohares info directly with decision 
making officers 

Identifies applications for refusals on 
based bona fides (i.e. intent to leave 
Canada, available funds) 

Bases risk patterns off applications 
with higher refusal rates (without 
documented adverse info) 

Finds all risk trends 
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Lighthouse dovetails with existing intelligence-based risk detection approaches. 
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Changed step 
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s.16(1)(b) 


Examples: 
Removal Orders 


Criminality 


Minimum number of 
| ! applications 
DATA MINING | : Minimum percent of 
| adverse 
Business Line 
Timeframe 


wh 
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out LH. 
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» L'évaluation de l'incidence algorithmique (EIA) pour Lighthouse a été élaborée par la Gestion des 


isques pour l'intégrité (GRI) et la Direction générale de la planification et du rendement des opérations 
(DGPRO); les intervenants d'IRCC ont été consultés. 


*Les repercussions de l'automatisation d'une decision administrative sont classées en quatre niveaux, allant du niveau I (incidence minime) au niveau IV 
(incidence très élevée). On distingue les niveaux d'incidence en fonction de critères de réversibilité et de durée attendue ; les décisions automatisées 
avant peu ou pas d'incidence sont reversibles et breves, tandis que celles ayant une incidence tres élevée sont irreversibles et perpetuelles. 


WESS & 


» 


. La prise de décisions continue d'étre la responsabilité des 


» Les principaux partenaires en matiere de données d'IRCC, le Systeme mondial de gestion des cas 
(SMGC) et l'entrepót de données d'entreprise (EDE), suivent des processus afin de s'assurer que seu 
des données pertinentes et fiables sont utilisées. Le SMGC comporte des processus rigoureux pour 
assurer la fiabilité des données utilisées dans le traitement des demandes. 


» 


existe un cadre de gouvernance pour Lighthouse qui décrit comment le systéme sera géré et 
a 


| 
maintenu de facon responsable. 
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s.21(1)(a) 
s.21(1)(b) 


. Plus, Diversity and Inclusion 


» IRM completed a GBA Plus analysis in May of 2022, the analysis assessed the potential for bias and 
differential impacts when using the Lighthouse risk identification tool. 

» If approved, the Methodology for Identifying Institutional Bias (Differential Outcomes Framework) will 
provide guidance to help ensure any bias is not perpetuated. 


Data Potential differential outcomes 
concern at the very front of the process, when an 
application is first received by IRC. O O O ŤűėáËĚËĚËaËaaaaaa 
—— ———— —M Refusals supported by Lighthouse are based on validated 
Refusal Likelihood — to avoid fettering of officer instances of fraud/criminality/malfeasance, not suspicion. 


Often, once significant fraud trends are interrupted, refusal 
reduced likelihood of being approved, the process is rates subsequently decrease, as knowledge amongst bad 
designed to ensure that decision-making officers will actors spreads that fraudulent applications will be 

not have access to Lighthouse-derived risk profiles. intercepted. 


decisions and any negative impact on the client via a 


Inequality (e.g., racism, sexism, etc.) — applications Md» | 
q y (e.g | dpp Applications matched by Lighthouse aligned well with the 


from different countries, gender, and age groups are 


treated in a manner generally consistent with the 
historical risk of adverse information being found in 
applications from those groups. 


underlying risk distribution in the general caseload. 


IRM will continue to monitor and review the tool's output for 
any real or perceived bias, and adjust the tool's parameters if 
unjust differential outcomes are observed. 
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s.21(1)(a) 
s.21(1)(b) 


En collaboration avec le CSA2 et le réseau intégré, la GRI devra : 


Mener des examens co ntinus des ra Dpo rts hebdomadaires sur les indicate u rs de ri sque fournis par 
le CSA2. L'examen servira de fonction de surveillance pour détecter tout biais non intentionnel dans 
les résultats de l'outil. 


Surveiller la pertinence des indicateurs de risque et en faire rapport au CSA2 en fonction de divers 
facteurs, comme le contexte, la mise à jour, la fréquence et les biais potentiels, au besoin 


o Un mécanisme de rétroaction sur les renseignements liés aux risques sera mis à la disposition 
des réseaux et d'autres intervenants pertinents afin que l'engagement et l'expertise continus 
soient partagés, et ce, pour veiller à ce que les configurations et les outils fonctionnent 
adéquatement et en conséquence. 
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s.21(1)(a) 
s.21(1)(b) 


rv : Dans le cadre de l'analyse continue de l'EIA et de l'ACS+, 
ngage 


ment externe aupres de partenaires clés, comme Statistique Canada. 
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gal considerations 
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Resources/cost 
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s.21(1)(a) 
s.21(1)(b) 


other AA tools 
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| Approval to launch 


| Concurrence with phased roll-out timelines 


a 9 


| Additional actions/miti 


gations/requirements id 
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